Natural language processing to identify lupus nephritis phenotype in electronic health records.

BACKGROUND: Systemic lupus erythematosus (SLE) is a rare autoimmune disorder characterized by an unpredictable course of flares and remission with diverse manifestations. Lupus nephritis, one of the major disease manifestations of SLE for organ damage and mortality, is a key component of lupus classification criteria. Accurately identifying lupus nephritis in electronic health records (EHRs) would therefore benefit large cohort observational studies and clinical trials where characterization of the patient population is critical for recruitment, study design, and analysis. Lupus nephritis can be recognized through procedure codes and structured data, such as laboratory tests. However, other critical information documenting lupus nephritis, such as histologic reports from kidney biopsies and prior medical history narratives, require sophisticated text processing to mine information from pathology reports and clinical notes. In this study, we developed algorithms to identify lupus nephritis with and without natural language processing (NLP) using EHR data from the Northwestern Medicine Enterprise Data Warehouse (NMEDW). METHODS: We developed five algorithms: a rule-based algorithm using only structured data (baseline algorithm) and four algorithms using different NLP models. The first NLP model applied simple regular expression for keywords search combined with structured data. The other three NLP models were based on regularized logistic regression and used different sets of features including positive mention of concept unique identifiers (CUIs), number of appearances of CUIs, and a mixture of three components (i.e. a curated list of CUIs, regular expression concepts, structured data) respectively. The baseline algorithm and the best performing NLP algorithm were externally validated on a dataset from Vanderbilt University Medical Center (VUMC). RESULTS: Our best performing NLP model incorporated features from both structured data, regular expression concepts, and mapped concept unique identifiers (CUIs) and showed improved F measure in both the NMEDW (0.41 vs 0.79) and VUMC (0.52 vs 0.93) datasets compared to the baseline lupus nephritis algorithm. CONCLUSION: Our NLP MetaMap mixed model improved the F-measure greatly compared to the structured data only algorithm in both internal and external validation datasets. The NLP algorithms can serve as powerful tools to accurately identify lupus nephritis phenotype in EHR for clinical research and better targeted therapies.

Helicobacter pylori Burden in the United States According to Individual Demographics and Geography: A Nationwide Analysis of the Veterans Healthcare System.

BACKGROUND & AIMS: There are no contemporary large-scale studies evaluating the burden of Helicobacter pylori in the United States according to detailed demographics. The primary objective was to evaluate H pylori positivity in a large national healthcare system according to individual demographics and geography. METHODS: We conducted a nationwide retrospective analysis of adults in the Veterans Health Administration who completed H pylori testing between 1999 and 2018. The primary outcome was H pylori positivity overall, as well as according to zip code-level geography, race, ethnicity, age, sex, and time period. RESULTS: Among 913,328 individuals (mean, 58.1 years; 90.2% male) included between 1999 and 2018, H pylori was diagnosed in 25.8%. Positivity was highest in non-Hispanic black (median, 40.2%; 95% confidence interval [CI], 40.0%-40.5%) and Hispanic (36.7%; 95% CI, 36.4%-37.1%) individuals and lowest in non-Hispanic white individuals (20.1%; 95% CI, 20.0%-20.2%). Although H pylori positivity declined in all racial and ethnic groups over the timeframe, the disproportionate burden of H pylori in non-Hispanic black and Hispanic compared with non-Hispanic white individuals persisted. Approximately 4.7% of the variation in H pylori positivity was explained by demographics, with race and ethnicity accounting for the vast majority. CONCLUSIONS: The burden of H pylori is substantial in the United States among veterans. These data should (1) motivate research aimed at better understanding why marked demographic differences in H pylori burden persist so that mitigating interventions may be implemented and (2) guide resource allocation to optimize H pylori testing and eradication in high-risk groups.

Team-Based Coaching Intervention to Improve Contrast-Associated Acute Kidney Injury: A Cluster-Randomized Trial.

BACKGROUND: Up to 14% of patients in the United States undergoing cardiac catheterization each year experience AKI. Consistent use of risk minimization preventive strategies may improve outcomes. We hypothesized that team-based coaching in a Virtual Learning Collaborative (Collaborative) would reduce postprocedural AKI compared with Technical Assistance (Assistance), both with and without Automated Surveillance Reporting (Surveillance). METHODS: The IMPROVE AKI trial was a 2×2 factorial cluster-randomized trial across 20 Veterans Affairs medical centers (VAMCs). Participating VAMCs received Assistance, Assistance with Surveillance, Collaborative, or Collaborative with Surveillance for 18 months to implement AKI prevention strategies. The Assistance and Collaborative approaches promoted hydration and limited NPO and contrast dye dosing. We fit logistic regression models for AKI with site-level random effects accounting for the clustering of patients within medical centers with a prespecified interest in exploring differences across the four intervention arms. RESULTS: Among VAMCs' 4517 patients, 510 experienced AKI (235 AKI events among 1314 patients with preexisting CKD). AKI events in each intervention cluster were 110 (13%) in Assistance, 122 (11%) in Assistance with Surveillance, 190 (13%) in Collaborative, and 88 (8%) in Collaborative with Surveillance. Compared with sites receiving Assistance alone, case-mix-adjusted differences in AKI event proportions were -3% (95% confidence interval [CI], -4 to -3) for Assistance with Surveillance, -3% (95% CI, -3 to -2) for Collaborative, and -5% (95% CI, -6 to -5) for Collaborative with Surveillance. The Collaborative with Surveillance intervention cluster had a substantial 46% reduction in AKI compared with Assistance alone (adjusted odds ratio=0.54; 0.40-0.74). CONCLUSIONS: This implementation trial estimates that the combination of Collaborative with Surveillance reduced the odds of AKI by 46% at VAMCs and is suggestive of a reduction among patients with CKD. CLINICAL TRIAL REGISTRY NAME AND REGISTRATION NUMBER: IMPROVE AKI Cluster-Randomized Trial (IMPROVE-AKI), NCT03556293.

The effect of telemental versus in-person mental health consults in the emergency department on 30-day utilization and processes of care.

OBJECTIVES: We sought to characterize how telemental health (TMH) versus in-person mental health consults affected 30-day postevaluation utilization outcomes and processes of care in Veterans presenting to the emergency department (ED) and urgent care clinic (UCC) with acute psychiatric complaints. METHODS: This exploratory retrospective cohort study was conducted in an ED and UCC located in a single Veterans Affairs system. A mental health provider administered TMH via iPad. The primary outcome was a composite of return ED/UCC visits, rehospitalizations, or death within 30 days. The following processes of care were collected during the index visit: changes to home psychiatric medications, admission, involuntary psychiatric hold placement, parenteral benzodiazepine or antipsychotic medication use, and physical restraints or seclusion. Data were abstracted from the Veterans Affairs electronic health record and the Clinical Data Warehouse. Multivariable logistic regression was performed. Adjusted odds ratios (aORs) with their 95% confidence intervals (95% CIs) were reported. RESULTS: Of the 496 Veterans in this analysis, 346 (69.8%) received TMH, and 150 (30.2%) received an in-person mental health evaluation. There was no significant difference in the primary outcome of 30-day return ED/UCC, rehospitalization, or death (aOR 1.47, 95% CI 0.87-2.49) between the TMH and in-person groups. TMH was significantly associated with increased ED/UCC length of stay (aOR 1.46, 95% CI 1.03-2.06) and decreased use of involuntary psychiatric holds (aOR 0.42, 95% CI 0.23-0.75). There were no associations between TMH and the other processes-of-care outcomes. CONCLUSIONS: TMH was not significantly associated with the 30-day composite outcome of return ED/UCC visits, rehospitalizations, and death compared with traditional in-person mental health evaluations. TMH was significantly associated with increased ED/UCC length of stay and decreased odds of placing an involuntary psychiatric hold. Future studies are required to confirm these findings and, if confirmed, explore the potential mechanisms for these associations.

Predictive models for abdominal aortic aneurysms using polygenic scores and PheWAS-derived risk factors.

Abdominal aortic aneurysms (AAA) are common enlargements of the abdominal aorta which can grow larger until rupture, often leading to death. Detection of AAA is often by ultrasonography and screening recommendations are mostly directed at men over 65 with a smoking history. Recent large-scale genome-wide association studies have identified genetic loci associated with AAA risk. We combined known risk factors, polygenic risk scores (PRS) and precedent clinical diagnoses from electronic health records (EHR) to develop predictive models for AAA, and compared performance against screening recommendations. The PRS included genome-wide summary statistics from the Million Veteran Program and FinnGen (10,467 cases, 378,713 controls of European ancestry), with optimization in Vanderbilt's BioVU and validated in the eMERGE Network, separately across both White and Black participants. Candidate diagnoses were identified through a temporally-oriented Phenome-wide association study in independent EHR data from Vanderbilt, and features were selected via elastic net. We calculated C-statistics in eMERGE for models including PRS, phecodes, and covariates using regression weights from BioVU. The AUC for the full model in the test set was 0.883 (95% CI 0.873-0.892), 0.844 (0.836-0.851) for covariates only, 0.613 (95% CI 0.604-0.622) when using primary USPSTF screening criteria, and 0.632 (95% CI 0.623-0.642) using primary and secondary criteria. Brier scores were between 0.003 and 0.023 for our models indicating good calibration, and net reclassification improvement over combined primary and secondary USPSTF criteria was 0.36-0.60. We provide PRS for AAA which are strongly associated with AAA risk and add to predictive model performance. These models substantially improve identification of people at risk of a AAA diagnosis compared with existing guidelines, with evidence of potential applicability in minority populations.

Maintaining a National Acute Kidney Injury Risk Prediction Model to Support Local Quality Benchmarking.

BACKGROUND: The utility of quality dashboards to inform decision-making and improve clinical outcomes is tightly linked to the accuracy of the information they provide and, in turn, accuracy of underlying prediction models. Despite recognition of the need to update prediction models to maintain accuracy over time, there is limited guidance on updating strategies. We compare predefined and surveillance-based updating strategies applied to a model supporting quality evaluations among US veterans. METHODS: We evaluated the performance of a US Department of Veterans Affairs-specific model for postcardiac catheterization acute kidney injury using routinely collected observational data over the 6 years following model development (n=90 295 procedures in 2013-2019). Predicted probabilities were generated from the original model, an annually retrained model, and a surveillance-based approach that monitored performance to inform the timing and method of updates. We evaluated how updating the national model impacted regional quality profiles. We compared observed-to-expected outcome ratios, where values above and below 1 indicated more and fewer adverse outcomes than expected, respectively. RESULTS: The original model overpredicted risk at the national level (observed-to-expected outcome ratio, 0.75 [0.74-0.77]). Annual retraining updated the model 5×; surveillance-based updating retrained once and recalibrated twice. While both strategies improved performance, the surveillance-based approach provided superior calibration (observed-to-expected outcome ratio, 1.01 [0.99-1.03] versus 0.94 [0.92-0.96]). Overprediction by the original model led to optimistic quality assessments, incorrectly indicating most of the US Department of Veterans Affairs' 18 regions observed fewer acute kidney injury events than predicted. Both updating strategies revealed 16 regions performed as expected and 2 regions increasingly underperformed, having more acute kidney injury events than predicted. CONCLUSIONS: Miscalibrated clinical prediction models provide inaccurate pictures of performance across clinical units, and degrading calibration further complicates our understanding of quality. Updating strategies tailored to health system needs and capacity should be incorporated into model implementation plans to promote the utility and longevity of quality reporting tools.

Associations Between Gastrointestinal Symptoms and COVID-19 Severity Outcomes Based on a Propensity Score-Weighted Analysis of a Nationwide Cohort.

Background and Aims: Gastrointestinal (GI) symptoms are well-recognized manifestations of coronavirus disease 2019 (COVID-19). Our primary objective was to evaluate the association between GI symptoms and COVID-19 severity. Methods: In this nationwide cohort of US veterans, we evaluated GI symptoms (nausea/vomiting/diarrhea) reported 30 days before and including the date of positive SARS-CoV-2 testing (March 1, 2020, to February 20, 2021). All patients had ≥1 year of prior baseline data and ≥60 days follow-up relative to the test date. We used propensity score (PS)-weighting to balance covariates in patients with vs without GI symptoms. The primary composite outcome was severe COVID-19, defined as hospital admission, intensive care unit admission, mechanical ventilation, or death within 60 days of positive testing. Results: Of 218,045 SARS-CoV-2 positive patients, 29,257 (13.4%) had GI symptoms. After PS weighting, all covariates were balanced. In the PS-weighted cohort, patients with vs without GI symptoms had severe COVID-19 more often (29.0% vs 17.1%; P < .001). When restricted to hospitalized patients (14.9%; n=32,430), patients with GI symptoms had similar frequencies of intensive care unit admission and mechanical ventilation compared with patients without symptoms. There was a significant age interaction; among hospitalized patients aged ≥70 years, lower COVID-19-associated mortality was observed in patients with vs without GI symptoms, even after accounting for COVID-19-specific medical treatments. Conclusion: In the largest integrated US health care system, SARS-CoV-2-positive patients with GI symptoms experienced severe COVID-19 outcomes more often than those without symptoms. Additional research on COVID-19-associated GI symptoms may inform preventive efforts and interventions to reduce severe COVID-19.

Information Extraction From Electronic Health Records to Predict Readmission Following Acute Myocardial Infarction: Does Natural Language Processing Using Clinical Notes Improve Prediction of Readmission?

Background Social risk factors influence rehospitalization rates yet are challenging to incorporate into prediction models. Integration of social risk factors using natural language processing (NLP) and machine learning could improve risk prediction of 30-day readmission following an acute myocardial infarction. Methods and Results Patients were enrolled into derivation and validation cohorts. The derivation cohort included inpatient discharges from Vanderbilt University Medical Center between January 1, 2007, and December 31, 2016, with a primary diagnosis of acute myocardial infarction, who were discharged alive, and not transferred from another facility. The validation cohort included patients from Dartmouth-Hitchcock Health Center between April 2, 2011, and December 31, 2016, meeting the same eligibility criteria described above. Data from both sites were linked to Centers for Medicare & Medicaid Services administrative data to supplement 30-day hospital readmissions. Clinical notes from each cohort were extracted, and an NLP model was deployed, counting mentions of 7 social risk factors. Five machine learning models were run using clinical and NLP-derived variables. Model discrimination and calibration were assessed, and receiver operating characteristic comparison analyses were performed. The 30-day rehospitalization rates among the derivation (n=6165) and validation (n=4024) cohorts were 15.1% (n=934) and 10.2% (n=412), respectively. The derivation models demonstrated no statistical improvement in model performance with the addition of the selected NLP-derived social risk factors. Conclusions Social risk factors extracted using NLP did not significantly improve 30-day readmission prediction among hospitalized patients with acute myocardial infarction. Alternative methods are needed to capture social risk factors.

Identifying Potential Therapeutic Applications and Diagnostic Harms of Increased Bilirubin Concentrations: A Clinical and Genetic Approach.

Bilirubin has antioxidant and anti-inflammatory properties in vitro and in animal studies and protects against inflammatory, cardiovascular, and other diseases in observational studies; therefore, bilirubin has potential as a therapeutic agent. However, observational studies could be confounded by many factors. We used a genetic (n = 61,281) and clinical (n = 234,670) approach to define the association between bilirubin and 19 conditions with a putative protective signal in observational studies. We also tested if individuals with genetically higher bilirubin levels underwent more diagnostic tests. We used a common variant in UGT1A1 (rs6742078) associated with an 26% increase in bilirubin levels in the genetic studies. Carriers of the variant had higher bilirubin levels (P = 2.2 × 10-16 ) but there was no significant association with any of the 19 conditions. In a phenome-wide association study (pheWAS) to seek undiscovered genetic associations, the only significant finding was increased risk of "jaundice-not of newborn." Carriers of the variant allele were more likely to undergo an abdominal ultrasound (odds ratio = 1.04, [1.00-1.08], P = 0.03). In contrast, clinically measured bilirubin levels were significantly associated with 15 of the 19 conditions (P < 0.003) and with 431 clinical diagnoses in the pheWAS (P < 1 × 10-5 adjusted for sex, age, and follow-up). With additional adjustment for smoking and body mass index, 7 of 19 conditions and 260 pheWAS diagnoses remained significantly associated with bilirubin levels. In conclusion, bilirubin does not protect against inflammatory or other diseases using a genetic approach; the many putative beneficial associations reported clinically are likely due to confounding.

Development of Electronic Health Record-Based Prediction Models for 30-Day Readmission Risk Among Patients Hospitalized for Acute Myocardial Infarction.

Importance: In the US, more than 600 000 adults will experience an acute myocardial infarction (AMI) each year, and up to 20% of the patients will be rehospitalized within 30 days. This study highlights the need for consideration of calibration in these risk models. Objective: To compare multiple machine learning risk prediction models using an electronic health record (EHR)-derived data set standardized to a common data model. Design, Setting, and Participants: This was a retrospective cohort study that developed risk prediction models for 30-day readmission among all inpatients discharged from Vanderbilt University Medical Center between January 1, 2007, and December 31, 2016, with a primary diagnosis of AMI who were not transferred from another facility. The model was externally validated at Dartmouth-Hitchcock Medical Center from April 2, 2011, to December 31, 2016. Data analysis occurred between January 4, 2019, and November 15, 2020. Exposures: Acute myocardial infarction that required hospital admission. Main Outcomes and Measures: The main outcome was thirty-day hospital readmission. A total of 141 candidate variables were considered from administrative codes, medication orders, and laboratory tests. Multiple risk prediction models were developed using parametric models (elastic net, least absolute shrinkage and selection operator, and ridge regression) and nonparametric models (random forest and gradient boosting). The models were assessed using holdout data with area under the receiver operating characteristic curve (AUROC), percentage of calibration, and calibration curve belts. Results: The final Vanderbilt University Medical Center cohort included 6163 unique patients, among whom the mean (SD) age was 67 (13) years, 4137 were male (67.1%), 1019 (16.5%) were Black or other race, and 933 (15.1%) were rehospitalized within 30 days. The final Dartmouth-Hitchcock Medical Center cohort included 4024 unique patients, with mean (SD) age of 68 (12) years; 2584 (64.2%) were male, 412 (10.2%) were rehospitalized within 30 days, and most of the cohort were non-Hispanic and White. The final test set AUROC performance was between 0.686 to 0.695 for the parametric models and 0.686 to 0.704 for the nonparametric models. In the validation cohort, AUROC performance was between 0.558 to 0.655 for parametric models and 0.606 to 0.608 for nonparametric models. Conclusions and Relevance: In this study, 5 machine learning models were developed and externally validated to predict 30-day readmission AMI hospitalization. These models can be deployed within an EHR using routinely collected data.

Validating a Computable Phenotype for Nephrotic Syndrome in Children and Adults Using PCORnet Data.

Background: Primary nephrotic syndromes are rare diseases which can impede adequate sample size for observational patient-oriented research and clinical trial enrollment. A computable phenotype may be powerful in identifying patients with these diseases for research across multiple institutions. Methods: A comprehensive algorithm of inclusion and exclusion ICD-9 and ICD-10 codes to identify patients with primary nephrotic syndrome was developed. The algorithm was executed against the PCORnet CDM at three institutions from January 1, 2009 to January 1, 2018, where a random selection of 50 cases and 50 noncases (individuals not meeting case criteria seen within the same calendar year and within 5 years of age of a case) were reviewed by a nephrologist, for a total of 150 cases and 150 noncases reviewed. The classification accuracy (sensitivity, specificity, positive and negative predictive value, F1 score) of the computable phenotype was determined. Results: The algorithm identified a total of 2708 patients with nephrotic syndrome from 4,305,092 distinct patients in the CDM at all sites from 2009 to 2018. For all sites, the sensitivity, specificity, and area under the curve of the algorithm were 99% (95% CI, 97% to 99%), 79% (95% CI, 74% to 85%), and 0.9 (0.84 to 0.97), respectively. The most common causes of false positive classification were secondary FSGS (nine out of 39) and lupus nephritis (nine out of 39). Conclusion: This computable phenotype had good classification in identifying both children and adults with primary nephrotic syndrome utilizing only ICD-9 and ICD-10 codes, which are available across institutions in the United States. This may facilitate future screening and enrollment for research studies and enable comparative effectiveness research. Further refinements to the algorithm including use of laboratory data or addition of natural language processing may help better distinguish primary and secondary causes of nephrotic syndrome.